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ABSTRACT 

Mathematics curricula are frequently rich with visuals, but these visuals are often not designed for 
optimal use of students ’ limited cognitive resources. The authors of this study revised the visuals in a 
mathematics lesson based on instructional design principles. The purpose of this study is to examine the 
effects of these revised visuals on students ’ cognitive load, cognitive processing, learning, and interest. 
Middle-school students (N = 62) read a lesson on early algebra with original or revised visuals while 
their eye movements were recorded. Students in the low prior knowledge group had less cognitive load 
and cognitive processing with the revised lesson than the original lesson. However, the reverse was true 
for students in the middle prior knowledge group. There were no effects of the revisions on learning. The 
findings are discussed in the context of the expertise reversal effect as well as the cognitive theory of 
multimedia learning and cognitive load theory. 

Keywords: Cognitive Load, Instructional Design Principles, Cognitive Processing, Multimedia Learning, 
Multiple Representations 

INTRODUCTION 

Eye-tracking measures may provide important insight into the design of learning 
materials (i.e., instructional design; Hyona, 2010; Mayer, 2010; van Gog & Scheiter, 2010). This 
view is based on the eye-mind assumption (Just & Carpenter, 1980), which states that the eye 
fixates on what the mind is processing (Just & Carpenter, 1976; Rayner, 1998). By examining 
what a student’s eyes fixate on, one can discern what that student is focusing on, and this 
information may be useful for understanding how students use instructional materials. 

One distinct benefit of eye-tracking measures is their spatial precision, which allows for 
understanding how information in different regions of a lesson is processed (e.g., Chang & Choi, 
2014; She & Chen, 2009). For this reason, eye-tracking measures are particularly valuable for 
understanding how different representations, such as visuals and text, are processed (e.g., Mason 
et al., 2013; Scheiter & van Gog, 2009; Schwonke, Berthold, & Renkl, 2009). In this research, 
we used eye tracking to examine how variations in visuals affect students’ processing of a lesson. 

Researchers and curriculum designers have articulated Instructional design principles 
(also called evidence-based principles and cognitive principles) that specify how visuals should 
be integrated with text (Mayer & Moreno, 1999; Mayer, 2008). The broad aim of these principles 
is to optimize learning (e.g., Mayer, 2009; Sweller, Ayres, & Kalyuga, 2011). In this work, we 
used eye tracking to examine students’ processing of a lesson that was either well aligned or less 
well aligned with these principles. 

To address this issue, we used a lesson from Connected Mathematics 2 ( CMP2; Lappan, 
Fey, Fitzgerald, Friel, & Phillips, 2006), which is rich with visuals, such as pictures, diagrams, 
and other spatial representations (Clinton, Cooper, Alibali & Nathan, 2012). However, the ways 
visuals are used in the lessons and activities do not always make effective use of students’ 



cognitive resources. In a separate, large-scale study, a team of researchers has revised the visuals 
based on instructional design principles, and is testing the revised version of the CMP2 
curriculum in a nation-wide randomized control trial in order to determine the effectiveness of 
the revisions on learning (Davenport, Kao, & Schneider, 2013). 

Building on previous research findings on instructional design principles and eye tracking 
(e.g., Johnson & Mayer, 2012; Ozcelik, Karakus, Kursun, & Cagiltay, 2009; Ozcelik, Arslan-Ari, 
Cagiltay, 2010), we conducted an eye-tracking experiment with students who read a lesson 
derived from the CMP2 curriculum with original visuals or with visuals that were revised on the 
basis of instructional design principles. The aim was to assess the effects of the revised visuals 
on students’ processing of the different representations and on their subsequent learning. 
Specifically, we were interested in how eye-tracking measures could reveal the moment-by¬ 
moment effort in working memory, referred to as cognitive load, as well as the amount of time 
spent viewing representations, referred to as the amount of cognitive processing (see Ozcelik et 
ah, 2010 for a similar approach). 

BACKGROUND 

The instructional design principles that guided the revisions are grounded in the cognitive 
theory of multimedia learning (Mayer, 2009; Mayer & Moreno, 2003) and in cognitive load 
theory (e.g., Paas, Renkl & Sweller, 2003; Plass, Moreno, & Briinken, 2010). A central idea of 
both theories is that the structure of the cognitive system imposes limits on the processing of 
information presented to auditory, linguistic, and visual sensory processing channels that 
influence how learners integrate information. The cognitive theory of multimedia learning holds 
that visual and verbal infonnation (i.e., text or speech) are processed in different pathways, and 
the theory emphasizes the need for the infonnation in these two pathways to be integrated 
(Mayer, 2014a). In addition, the cognitive theory of multimedia learning prescribes guidance for 
instructional design, namely the reduction of extraneous (i.e., unnecessary) processing to 
improve learning (Mayer, 2009). In sum, implementing these theory-based principles in a lesson 
should reduce the amount of cognitive processing necessary to understand a lesson. 

In contrast to the cognitive theory of multimedia learning, cognitive load theory 
emphasizes the different types of cognitive load (i.e., effort in working memory) a student may 
experience (Sweller et ah, 2011). Cognitive load theory builds on the idea that inherent (i.e., 
biological) constraints on working memory limit the amount of information one can process at a 
given time (Chandler & Sweller, 1991). If a student has too much infonnation to process or 
information is difficult to understand, limited working memory capacity can be overloaded, 
thereby impairing comprehension and diminishing learning (Sweller, 1994). Much of this 
research has shown superior problem-solving perfonnance and learning with manipulations that 
reduce the cognitive load required to integrate information across visuals and text (e.g., Chandler 
& Sweller, 1991; Kalyuga, Chandler, & Sweller, 1999; see Mayer & Moreno, 2009; Pashler et 
al., 2007 for reviews). 

Cognitive load can be intrinsic or extrinsic to the learning goal (Sweller et al., 2011). 
Intrinsic cognitive load (this includes what used to be referred to as “germane load”; Kalyuga, 
2011) consists of the infonnation in working memory relevant to the task. In contrast, extrinsic 
cognitive load consists of information in working memory inelevant to the instructional task. 

The general aim of instructional design principles is to reduce extrinsic cognitive load so that 
students can focus their cognitive resources to best manage the lesson content (Mayer & Moreno, 
2003). 



Instructional design principles 

Three instructional design principles guided the revisions to the curricular materials: 
signaling, contiguity, and coherence. 

According to the signaling principle, learning is promoted by cues, such as color codes 
and labels (Mayer, 2009), to important information. Cues may promote learning both by 
directing students’ attention to relevant infonnation and by connecting corresponding 
information across different representations (e.g., text and visuals; Berthold & Renkl, 2009; 
Florax & Ploetzner, 2010; Kalyuga, Chandler & Sweller, 1999; de Koning, Tabbers, Rikers, & 
Paas, 2009; for a meta-analysis see Richter, Scheiter & Eitel, 2016). These cues may reduce the 
extraneous processing and the extrinsic cognitive load needed to discern the importance of 
information or integrate corresponding infonnation across representations (Lin & Lin, 2014; 
Mayer & Moreno, 2003). The use of signaling may increase cognitive processing of the visual, 
especially in the signaled areas of the visual (de Koning, Tabbers, Rikers, & Paas, 2010). 
However, previous eye-tracking findings have not indicated a reduction in cognitive load due to 
signaling (de Koning et ah, 2010; Ozcelik et ah, 2010). 

The contiguity principle states that infonnation should be ananged such that relevant 
information in different representations is in close proximity. This reduces the cognitive load of 
reading and connecting conesponding verbal and visual infonnation (e.g., Ginns, 2006; 

Renshaw, Finlay, Tyfa, & Ward, 2004). Labels place relevant text in close proximity to visuals, 
making the information in the two representations more spatially contiguous, thereby facilitating 
integration between the two representations (Florax & Ploetzner, 2010; Holsanova, Holmberg, & 
Hohnquist, 2009; Johnson & Mayer, 2012). Previous eye-tracking findings have indicated that 
integrating relevant text with visuals does not affect the amount of cognitive processing of the 
visual, but does reduce the amount of cognitive processing of the text, perhaps due to the 
reduction in extraneous processing (Johnson & Mayer, 2012). However, there has been limited 
empirical investigation of the contiguity principle on cognitive load (see Altan & Cagiltay, 2015, 
for preliminary work with a small sample). 

The coherence principle states that learning is fostered when interesting, but irrelevant 
information, such as decorative pictures, is removed (Harp & Mayer, 1997; Mayer, 2009; for a 
review, see Rey, 2012). This type of infonnation has been found to distract learners and diminish 
comprehension, a phenomenon referred to as the seductive details effect (e.g., Lehman, Schraw, 
McCrudden, & Hartley, 2007). Irrelevant visuals may interfere with learning because they 
increase the amount of infonnation in the lesson (Sanchez & Wiley, 2006). This potential 
increase in extraneous cognitive processing and extrinsic cognitive load appears to be 
particularly problematic for learning from written lessons in which verbal infonnation is 
conveyed through text (visually) compared to oral presentations in which verbal infonnation is 
conveyed through nanation (auditorily; Park, Moreno, Seufert, & Briinken, 2011). This 
difference in modalities is presumably due to irrelevant visuals overloading cognitive resources 
when all of the information in a lesson is presented visually (Park, Flowerday, & Briinken, 

2015). Previous eye-tracking findings have indicated that irrelevant visuals in statistics lessons 
caused less cognitive processing of instructional text and visuals compared to lessons without 
irrelevant visuals, likely because of the extra information and distraction of the irrelevant visuals 
(Rey, 2014). Surprisingly, one study found that self-reports of cognitive load were lower for 
lessons with irrelevant visuals, despite lower comprehension scores (Park, Korbach, & Briinken, 
2015. It is possible that the irrelevant visuals gave students the impression the lesson was easier 
than it actually was. For this reason, an eye-tracking measure of cognitive load may be 



particularly valuable (van Gog, Kester, Nievelstein, Giesbers, & Paas, 2009) for materials such 
as these. An eye-tracking measure would be collected as the lesson is being viewed and would be 
a more objective measure of the demand imposed by the lesson (e.g., Amadieu, Van Gog, Paas, 
Tricot, & Marine, 2009). 

In past research, the signaling, contiguity, and coherence principles have typically been 
examined in isolation (e.g., Florax & Ploetzner, 2010; Mason et ah, 2013; Ozcelik et ah, 2009, 
2010; Scheiter & Eitel, 2015). However, in the revisions of the CMP curriculum, these principles 
were all applied in the following ways: (1) additional signaling was added; (2) contiguity of 
visuals and related text was increased, and (3) math-irrelevant visuals were removed. Multiple 
principles were applied based on a “less is more” approach; less extrinsic cognitive load and 
extraneous processing (through the removal of math-irrelevant visuals and increased contiguity 
and signaling) was expected to yield more learning (see Mayer, 2014b). This approach is novel 
and examines whether benefits to the process and products of learning can be maximized by 
applying multiple principles simultaneously. 

Instructional design principles can interact with prior knowledge 

Generally speaking, the cognitive load and cognitive processing involved when learning 
from lessons with text and visuals varies with both the design of the material (as previously 
discussed) and the prior knowledge of the student (Kalyuga et ah, 2003; Kirschner, Paas, 
Kirschner, & Janssen, 2011; Moreno, 2004). Some evidence suggests that the implementation of 
instructional design principles may be most effective for students with low levels of prior 
knowledge (Mayer, 2001). This is because the less prior knowledge a student has, the more 
intrinsic cognitive load the task imposes (Kalyuga, 2011; Leahy, Hanham, & Sweller, 2015). An 
increase in intrinsic cognitive load could consume working memory capacity, leaving little 
capacity for handling extrinsic cognitive load (Paas, Renkl, & Sweller, 2003). For these reasons, 
if a task is high in intrinsic cognitive load, reductions in extrinsic load should yield more benefits 
compared to tasks low in intrinsic load (Seufert, Janen, & Brunken, 2007). Therefore, the 
reduction of extraneous processing and extrinsic cognitive load through the application of the 
instructional design principles may foster greater learning in students with low levels of prior 
knowledge than in students with high levels of prior knowledge (e.g., Magner, Schwonke, 
Aleven, Popescu & Renkl, 2014; see Moreno & Mayer, 2007; Schnotz, 2002, for discussions). It 
is also possible that efforts to lower cognitive load may actually make learning more difficult for 
students with higher levels of prior knowledge (Kalyuga, 2007). This phenomenon, known as the 
expertise reversal effect, arises because the information added to guide processing is redundant 
with what students with high prior knowledge already know, thereby increasing the cognitive 
load of the lesson (Sweller et ah, 2011). 

Much of the previous work on prior knowledge and instructional design techniques has 
involved separating students into two groups: high and low (e.g., Mayer & Gallini, 1990; Mayer, 
Steinhoff, Bower, & Mars, 1995). However, recent research findings have indicated that 
separating students into three groups allows a more nuanced understanding of interactions 
between instructional design techniques and prior knowledge (Magner et al., 2014). 

THE CURRENT STUDY 

The purpose of the current study is to use eye-tracking methodology to examine the 
effects of revisions to visuals based on instructional design principles (specifically, the signaling, 
contiguity, and coherence principles) on the process of reading a mathematics lesson and on 
subsequent learning from that lesson. Eye tracking was the methodology of choice because the 
data from eye tracking can be used to infer the moment-by-moment processes involved in 



reading (Just & Carpenter, 1980; Rayner, 1998). Because of the spatial precision of eye tracking, 
the data afford valuable insight into the processing of different representations in written lessons 
with visual representations (e.g.. Mason et ah, 2013; Rau, Michaelis, & Fay, 2015; Scheiter & 
Eitel, 2015; see Hyona, 2010 for review). In other words, eye tracking allows an examination of 
how visuals and text are processed. 

Specific to this study, we were interested in how eye tracking could yield information 
about the cognitive load and amount of cognitive processing involved in viewing the text and 
visuals. To assess cognitive load, average fixation length (i.e., pause in eye movement) was used. 
Average fixation length is thought to be a positive indicator of cognitive load (i.e., as cognitive 
load increases, average fixation length increases; Ozcelik et al. 2010; van Gog et ah, 2009; Paas, 
2009; van Gog & Scheiter, 2010). This is because cognitive load is essentially the mental effort 
involved in working memory, and fixations typically increase as more effort is exerted while 
viewing the fixated material (van Gog et al., 2009). Thus, average fixation length can be used to 
infer the effectiveness of the revisions in reducing cognitive load. In contrast, the amount of 
cognitive processing is how much time one spends thinking about something. Given the eye- 
mind assumption (Just & Carpenter, 1980), the time spent viewing a representation, calculated 
by summing the fixation durations on that representation, is considered to be the amount of 
cognitive processing involved with the representation (e.g., Graesser, Lu, Olde, Cooper-Pye, & 
Whitten, 2005; Rayner, 1998; Ozcelik et al., 2009). In other words, the amount of time a student 
viewed a section of a lesson provides a measure of how much that student cognitively processed 
that section (e.g., Kaakinen, Olkoniemi, Kinnari, & Hyona, 2014). The amount of cognitive 
processing differs from cognitive load because cognitive processing is the overall time spent on a 
representation whereas cognitive load is the mental effort in working memory at a given moment 
(Ozcelik et al., 2010). 

We address three main research questions. First, what were the effects of the revisions on 
cognitive load while reading the lesson? Because the revisions were designed to reduce cognitive 
load, average fixation length was expected to be shorter for the revised lesson than the original 
lesson (e.g., Amadieu et al., 2009). In this way, average fixation length can be used to evaluate 
the effectiveness of the revisions. Further, this finding might be strongest for students with low 
levels of prior knowledge, compared to students with higher levels of prior knowledge (Mayer, 
2001). Moreover, because of the expertise reversal effect (Kalyuga, 2007), it is possible that the 
revisions would increase the cognitive load for students with middle and high levels of prior 
knowledge. It should be noted that some previous eye-tracking work examining a single 
instructional design principle has not revealed effects on average fixation length (Altan & 
Cagiltay, 2015; de Koning et al., 2010; Ozcelik et al., 2010). However, because our study applies 
multiple principles, the cumulative effects of these principles may be powerful enough to reduce 
average fixation length, at least for students with low levels of prior knowledge. 

Second, what were the effects of the revisions on the amount of cognitive processing for 
different representations? These amounts can be assessed via total fixation time, which is the 
summed duration of all fixations within an area of interest (e.g., a visual or a section of text;). 
With this measure, it can be determined whether students differed in how much they processed 
the original and revised visuals. Given that the revised visuals provide additional guidance for 
processing, students may need to engage in less cognitive processing with them, thereby needing 
less total fixation time compared to the original visuals. In addition, it can be determined whether 
students processed the text in the lesson differently depending on the type of visuals used. 
Students with revised visuals may have less total fixation time on the text compared to students 



with original visuals because students with revised visuals may need less help from the text to 
understand the mathematical content. Finally, to assess the amount of cognitive processing of the 
entire lesson, we examined the total fixation duration for the entire lesson. It was expected that 
students with the revised lesson would spend less time with the lesson than students with the 
original lesson. 

As with cognitive load, the effects of the revisions on the amount of cognitive processing 
would likely be most pronounced for students with low levels of prior knowledge. However, 
prior knowledge is especially important to consider given that the amount of cognitive 
processing of visuals is negatively associated with prior knowledge (i.e., the less prior 
knowledge a student has, the longer the visual is viewed; Hegarty & Just, 1993; Schwonke et ah, 
2009). This may be because students with high prior knowledge primarily use the visuals to 
confirm what they already know, rather than to leam from them (Rasch & Schnotz, 2009). 

Third, did the revisions of the visuals affect students’ learning from the lesson? The 
instructional design principles were intended to reduce cognitive load and extraneous processing 
so that students could focus their limited cognitive resources on the content presented in the 
lesson (Mayer, 2009; Sweller et al., 2011). For these reasons, students may perform better on a 
post-lesson test after reading the revised lesson than the original lesson. However, the learning 
benefits from the revisions may be strongest for students with low levels of prior knowledge 
(Mayer, 2001). 

One benefit of eye tracking is that it can reveal whether the revisions affected how 
students viewed the lesson, even if there is no observable effect on learning. For example, 
previous eye-tracking findings have revealed that replacing text with narration in a multimedia 
presentation reduced average fixation lengths when viewing the presentation, but did not affect 
learning from the presentation (Liu, Lai, & Chuang, 2011). This is because eye-tracking 
measures are more sensitive than many other measures. Furthermore, the amount of cognitive 
processing of the representations in the lesson could be used to examine if students compensate 
for the lack of guidance in the original visuals by spending more time on different 
representations in the lesson as well on the lesson overall. Finally, if the revisions had no effect 
on eye movements or learning, it can be assumed that the application of the instructional design 
principles was ineffective and other techniques should be explored (e.g., Lowe & Boucheix, 
2011 ). 

These questions were addressed using a lesson about how to graph independent and 
dependent variables on a coordinate grid. In addition to being visually rich, this topic is of 
particular interest because of its importance in scientific literacy (Padilla, McKenzie, & Shaw, 
1986) and algebraic reasoning (Nathan & Kim, 2007). 

Method 

Participants 

Participants were 62 (26 female, 36 male) middle-school students entering sixth or 
seventh grade (ages 10 - 12 years; M = 11.12 years, SD = .33 years). Because of apparatus 
malfunction, eye-tracking data were collected for only 57 participants. However, all 62 
participants read the lesson in the same manner and completed the self-report of prior knowledge 
and the post-lesson test (see Measures). Participants were compensated with a $15 gift card for 
an online retailer. 

Apparatus 

An EyeLink 1000 Desk-Mounted System, manufactured by SR Research Ltd. (Toronto, 
Ontario, Canada), was used to collect eye movement data. This eye tracker uses an infra-red 



video camera for monocular tracking, and the video camera was focused on the participant’s 
pupil. The video camera sampled real-time fixations at a 1000 Hz sampling rate. Head position 
was stabilized with a chin and forehead rest 70 cm from the computer monitor displaying the 
lesson. Pupil diameter was recorded with centroid pupil tracking. 

Materials 

The lesson, derived from CMP2 (Lappan et ah, 2006), covered the skills necessary to 
record data with independent and dependent variables in a table and then construct a graph from 
those data. The skills were presented in the context of a story in which a person was planning a 
long-distance bike trip and needed to know her biking pace. The original lesson included a 
variety of visuals including a map, graphs, tables, and math-irrelevant pictures. 

The lesson consisted of nine pages, with identical text in both the original and revised 
conditions but changes made to the visuals on eight of the revised pages. The first page was the 
same for both groups, with text that reiterated the instructions for reading the lesson (e.g., read at 
your own pace, please sit still). The introduction to the lesson discussed the person’s plans for 
her trip and included a map that had decorative features in the original materials, but, based on 
the coherence principle, did not have decorative features in the revised materials. In the 
remaining 7 pages, there were 3 pages in which the signaling and contiguity principles were 
applied to the revised visuals. Specifically, color coding, labels, and call-out boxes were added to 
the math-relevant visuals of tables and graphs (see Figure 1). There were 2 pages in which the 
math-relevant visuals were the same in the original and revised materials, but the original 
materials also contained math-irrelevant visuals and, following the coherence principle, these 
were deleted in the revised materials (see Figure 2). Finally there were 2 pages in which the 
original materials contained a math-irrelevant visual and, following the signaling, contiguity, and 
coherence principles, the revised materials contained a math-relevant visual with labeling and 
call-out boxes (see Figure 3). 

As one would likely see in CMP2, the arrangement of the visuals in relationship to the 
text varied on each page. On some pages the visuals were below the text and on other pages the 
visuals were beside the text. This created more variability in the design of the materials across 
the lesson than is typically seen in eye-tracking experiments of lessons with visuals (e.g., 

Scheiter & Eitel, 2015). However, the aim was to make materials as authentic as possible to 
enhance the ecological validity of the results (see Holsanova, 2014). 
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To plot this information on a coordinate graph, start at the 
-v-vaJuc of 2 on the A-axis (time) and follow a line straight 
up. On thc /-axis (distance), start at the /-value of 22 and 
follow a line straight across. Make a point where the two 
lines intersect. You can describe this point with the 
coordinate pair (2,22). The first number in a coordinate 
pair is the x-coordinatc, and the second number is the 
v -cot udinate 
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To plot this information on a coordinate graph, start at 
the a- value of 2 on the A-axis (time) and follow a line straight 
up. On the /-axis (distance), start at the /-value of 22 and 
follow a line straight across. Make a point where the two 
lines intersect. You can describe this point with the 
coordinate pair (2,22). The first number in a coordinate 
pair is the .v-coordinate, and the second number is the 
^coordinate. 



Figure 1. Example of revisions based on the signaling and contiguity principles 
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Revised 


Recording Data and Making Tables 

Celia took a practice ride in her hometown of Atlantic City. She used an app on her 
smart phone that recorded how far she had hiked ever)' half hour. She made the table 
below of the distance in miles she had biked every half hour. 
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(miles) 
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18 

22 

26 

30 

34 

40 

45 

49 

52 

58 


After the first half hour (0.5 hours), she had biked for seven miles. At four hours, she 
had biked for 40 miles. At the end of the day, she had biked six hours for a total of 
58 miles. 



Recording Data and Making Tables 

Celia took a practice ride in her hometown of Atlantic City. She used an app on her 
smart phone that recorded how far she had biked every half hour. She made the table 
below of the distance in miles she had biked every half hour. 
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After the first half hour (0.5 hours), she had biked for seven miles. At four hours, she 
had biked for 40 miles. At the end of the day, she had biked six hours for a total of 
58 miles. 


Figure 2. Example of revisions based on the coherence principle (image from Shutterstock©, 
used with permission) 
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Step 5 Finish plotting the data points. Continue plotting the information in the data table on the coordinate graph. 



Step 5 Finish plotting the data points. Continue plotting the information in the data table on the coordinate graph. 
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Figure 3. Example of revisions based on the coherence, signaling, and contiguity principles 
(image in original lesson from Getty Images©, used with permission) 

Procedure 

Participants took part individually, and the experimenter calibrated the eye tracker for 
each participant. Participants then read the lesson one page at a time, at their own pace, while 
their eye movements were recorded. Prior to reading each page, participants gazed at a single dot 
on the screen to correct for drifts in eye gaze that may have occurred since calibration. After 
reading the lesson, participants reported their familiarity with the math content and then 
completed the post-lesson test. 

Measures 

Similar to Mautone and Mayer (2001), prior knowledge was assessed through self-reports 
of familiarity with the mathematical content of the lesson on a Likert scale from 1-5 (M= 3.74, 
SD = 1.20). Based on these responses, participants were divided into three prior knowledge 
groups: low (responses between 1-3; n = 22), middle (response of 4; n = 16), and high (response 
of 5; n = 19). Self-reports were used instead of pretests to prevent pretest sensitization, in which 
treatment effects may be inflated by priming knowledge with assessments prior to the lesson 
(Willson & Putnam, 1982; Willson & Kim, 2010). 

The post-lesson test consisted of three parts. The first part directly assessed individual 
skills covered in the lesson, such as reading data values from a table and graph, identifying 
independent and dependent variables, identifying appropriately scaled axes, using appropriate 
axis units and scales, and locating the x- and y-axes. Answers on the first part were scored for 
accuracy (0 for incorrect answers; 1 for correct answers). The second part asked participants to 
construct a graph based on a table of data points. Students’ graph construction was scored out of 

5 points based on variable and axis placement, consistent use of scale on each axis, variable 
labels, and plotting points. The third part assessed preparation for future learning (see Bransford 

6 Schwartz, 1999) by presenting a short, novel lesson that students read and answered questions 
about. The preparation for future learning questions asked students to take their reasoning a step 
further and to match a graph’s pattern to a description of a data scenario. As with the first part, 
























accuracy for question answers was dichotomously scored. A total score was determined by 
summing scores on all three parts (maximum of 27 points). 

RESULTS 

For all analyses, the Type I error rate was set at a = .05, with Bonferroni corrections for 
multiple comparisons. 

What were the effects of the revisions on cognitive load while reading the lesson? 

We first examined the cognitive load incurred while reading the text, as indicated by 
average fixation length. Eye-tracking measures were extracted from text at the sentence level. 
Following Blozis and Traxler (2007), linear mixed-model analyses with condition and prior 
knowledge group as fixed factors, participant as a random factor, sentence as a repeated effect, 
and average fixation length as the dependent variable were conducted (see Schonberg, 

Sandhofer, Tsang, & Johnson, 2014, for similar analyses). All lesson text after the first page of 
instructions was used in these analyses. 

As predicted, there was an interaction between condition and prior knowledge group, 

F( 2, 51) = 47.27,/? < .001 (see Figure 4). Participants in the low prior knowledge group had 
longer average fixation lengths on the text in the original condition than the revised condition, 
F(l, 21) = 45.42,/? < .001. In contrast, participants in the middle prior knowledge group had 
shorter average fixation lengths on the text in the original condition than the revised condition, 

F( 1, 15) = 71.15,/? < .001. Participants in the high prior knowledge group also had shorter 
average fixation lengths on the text in the original condition than the revised condition, but this 
difference did not reach significance, F(l, 18) = 3.613,/? = .06. These findings indicate that the 
cognitive load involved when reading the text varies as a function of both prior knowledge and 
revisions to the visuals. 

In addition to the significant interaction, there was also a main effect for condition, F( 1, 
51) = 7.75, /? = .01. Average fixation length was shorter overall while viewing text in the original 
condition than the revised condition, because participants in both the middle and high prior 
knowledge groups had shorter average fixation lengths in the original condition. There was also 
a main effect for prior knowledge, F( 2, 51) = 52.41 p= .01, with participants in the high prior 
knowledge group having shorter average fixation lengths than both participants in the middle 
prior knowledge group,/? = .001, and participants in the low prior knowledge group,/? < .001. In 
addition, participants in the middle prior knowledge group had shorter average fixation lengths 
than did participants in the low prior knowledge group,/? < .001. Overall, the decrease in average 
fixation duration with higher levels of prior knowledge indicated that cognitive load was lower 
for students with more knowledge. This relationship between prior knowledge and cognitive load 
is consistent with previous findings (Kalyuga, 2011) and validates the use of self-reports for 
assessing prior knowledge. 




Figure 4. Average fixation length (means and +/1 SE) by condition and prior knowledge group 
while reading the lesson text 

We next examined the effects of the revisions on the cognitive load incurred while 
processing math-relevant visuals (i.e., graphs and tables). To do this, the average fixation lengths 
on the math-relevant visuals that were present in both the original and revised lessons were 
examined (see Table 1 for an example of math-relevant visuals). The same analyses as with text 
were conducted, except the dependent variable was the average fixation length on math-relevant 
visuals. Similar to the text analyses, there was an interaction of condition and prior knowledge 
on average fixation length on the math-relevant visuals, F( 2, 51) = 8.58,/? < .001 (see Figure 2). 
For participants in the low prior knowledge group, the average fixation length for math-relevant 
visuals was longer in the original condition than in the revised condition F( 1,21) = 5.53,/? = .02. 
In contrast, for the middle prior knowledge group, the average fixation length for math-relevant 
visuals was shorter in the original condition than in the revised condition, TfiT, 15) = 9.77,/? = 
.003. Finally, for the high prior knowledge group, the average fixation length for math-relevant 
visuals was longer in the original condition than in the revised condition, F(\ , 18) = 9.29, p = 
.003. There was no overall main effect of condition, and there was a marginally significant main 
effect of prior knowledge, F( 2, 51) = 2.80,/? = .06, with average fixation length generally being 
shorter with higher levels of prior knowledge. 
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Figure 5. Average fixation length (means and +/1 SE) by condition and prior knowledge group 
while viewing original or revised math-relevant visuals 


What were the effects of the revisions on the amount of cognitive processing for different 
representations in the lesson? 

The effects of the revisions to the visuals on the amount of cognitive processing of the 
text and the visuals were examined. Total fixation duration (i.e., the sum of all fixations on a 
region of interest) was the measure of amount of cognitive processing. Prior to analyses, total 
fixation time was logarithmically transformed to improve nonnality (e.g., Mason, Tornatora, & 
Pluchino, 2015). Analyses were similar to those for cognitive load except the dependent variable 
was the total fixation duration on the region of interest. 

We also examined the effects of the revisions on the cognitive processing of the text. 
Results are shown in Figure 6 (actual, non-transformed total fixation time is presented in the 
graphs to provide the reader with meaningful descriptive statistics). There was an interaction 
between condition and prior knowledge group for total fixation duration on the text, F( 2, 51) = 
23.63 ,p < .001. Participants in the low prior knowledge group had greater total fixation duration 
on the text in the original conditions than the revised condition, F(l, 21) = 74.88, p < .001. In 
contrast, participants in the middle prior knowledge group had less total fixation duration on the 
text in the original condition than the revised condition, F( 1, 15) = 9.82, p = .002. There was no 
effect of condition on total fixation duration on the text for participants in the high prior 
knowledge group. 

In addition to this interaction, there was also a main effect of condition, F( 1, 51) = 7.58, 
= .01, with greater total fixation duration on the text in the original condition than the revised 
condition, which is likely due to the reading behavior of participants in the low prior knowledge 
group. In addition, there was a main effect of prior knowledge, F( 2, 51) = 49.02, p < .001. 
Participants in the low prior knowledge group had greater total fixation duration on the text than 
did both participants in the middle prior knowledge group ,p< .001, and participants in the high 
prior knowledge group, p < .001. 
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Figure 6. Total fixation duration (means and +/1 SE) by condition and prior knowledge group 
while reading the lesson text 


We also examined the effects of the revisions on the cognitive processing of math¬ 
relevant visuals. Results are shown in Figure 4. 

There was an interaction between condition and prior knowledge group for total fixation 
duration on the math-relevant visuals, F( 2, 51) = 6.57, p = .002 (see Figure 7). Participants in the 
low prior knowledge group had longer total fixation duration on the math-relevant visuals in the 
original condition than in the revised condition, F(l, 21) = 6.43, p = .01. In contrast, participants 
in the middle prior knowledge group had shorter total fixation duration on the math-relevant 
visuals in the revised condition than in the original condition, F( 1, 15) = 9.74,/? = .003. There 
was no effect of condition on total fixation duration on the math-relevant visuals for participants 
in the high prior knowledge group. Therefore, revisions appeared to reduce cognitive processing 
for students in the low prior knowledge group, but increased cognitive processing for students in 
the middle prior knowledge group (explanations for these findings will be considered in the 
discussion). There was no main effect of condition, but there was a main effect of prior 
knowledge group, F( 2, 51) = 9.70,/? < .001. Participants in the low prior knowledge group had 
longer total fixation duration on math-relevant visuals than did either participants in the middle 
prior knowledge group,/? = .002, or the high prior knowledge group,/? < .001. 
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Figure 7. Total fixation duration (means and +/1 SE) by condition and prior knowledge group 
while viewing original or revised math-relevant visuals 


Finally, the amount of cognitive processing of the entire lesson was examined (termed 
time spent with lesson to distinguish from previous analyses). An ANOVA with condition and 
prior knowledge group as independent variables and the total number of seconds spent with the 
lesson as the dependent variable was conducted. Results are shown in Figure 8. Condition and 
did not affect the time spent with the lesson, F(l, 51) = .36, p = .55 and there was a marginal 
effect of prior knowledge group, F( 2, 51) = 2.91, p = .06, with time with the lesson generally 
being less with more prior knowledge. There was no reliable interaction between condition and 
prior knowledge group, F(2, 51) = 2.22, p = . 12, although, based on the means, the revised lesson 
appeared to be read more quickly than the original lesson for the low prior knowledge group. 
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Figure 8. Time spent with lesson (means and +/1 SE) by condition and prior knowledge group 


Did the revisions affect learning? 

We examined the effects of the revisions on learning from the lesson. An ANOVA with 
condition and prior knowledge group as independent variables and score on the post-lesson test 
as the dependent variable was conducted. For these analyses, data from all participants, not just 
those for whom eye-tracking data were recorded, are reported (N = 62). As seen in Figure 9, 
condition did not affect post-lesson test scores, F(\ , 56) = .00 ,p = .96. However, level of prior 
knowledge did predict post-lesson test scores, F(2, 56) = 10.19,/? < .001; pairwise comparisons 
revealed that participants in the low prior knowledge group had lower post-lesson test 
performance than did participants in the high prior knowledge group, p < .001, Cohen’s d= 1.30, 
providing validity for use of the self-report as a prior knowledge measure. There was no 
interaction between condition and level of prior knowledge, F( 2, 56) = .14,/? = .87. 
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Figure 9. Post-lesson scores (means and +/1 SE) by condition and prior knowledge group 


DISCUSSION 

This study examined the effects of revisions of mathematical lesson materials based on 
instructional design principles on measures of cognitive load, amount of cognitive processing, 
and learning. The findings indicated that the revisions affected cognitive load and amount of 
cognitive processing for visuals and text, but that the effect of the revisions varied with students’ 
level of prior knowledge. No effects of the revisions were found for amount of processing of the 
lesson as a whole or learning from the lesson. 

Cognitive load 

It was expected that the revisions would reduce cognitive load, as indicated by average 
fixation length, especially for students with low levels of prior knowledge. Indeed, the revisions 
appeared to reduce cognitive load while reading the text for students with low levels of prior 
knowledge. In addition, the revisions appeared to reduce cognitive load for viewing the revised 
visuals for students in the low prior knowledge group and high prior knowledge group. These 
findings support the idea that instructional design principles can reduce the cognitive load of 
reading text and attending to math-relevant visuals, at least for students with low levels of prior 
knowledge. 

However, the revisions increased the cognitive load of reading the lesson text and 
viewing the math-relevant visuals for students with moderate prior knowledge. These findings 
are somewhat consistent with the expertise reversal effect, a phenomenon in which guidance 
intended to assist learning actually makes learning more difficult for students with higher levels 
of prior knowledge (Kalyuga, 2007). This is because the guidance is unnecessary infonnation for 
someone who is proficient with the material. Specifically, the labeling of revised visuals may 
have been redundant information that required extra processing and thus increased cognitive load 
for the students with more knowledge (Kalyuga, Ayres, & Sweller, 2003). For students in the 
high prior knowledge group, this trend was reversed when these students were viewing the math- 


relevant visuals. One speculative explanation, based on previous work on how students with high 
levels of prior knowledge use visuals (Rasch & Schnotz, 2009) is that these students had such 
proficiency with the lesson that they were able to use the visuals to help remind them of the 
material they already knew. However, it is worth noting that this pattern, in which students with 
the highest level of prior knowledge have a reversal of the expertise reversal effect, has not been 
previously noted in the literature; therefore, it should be interpreted with caution. 

Cognitive processing 

It was expected that amount of cognitive processing of the text and visuals, as indicated 
by total fixation duration, would be less for the revised lesson than the original, at least for 
students with low levels of prior knowledge. This is because, based on the cognitive theory of 
multimedia learning, the revisions should have reduced extraneous processing (Mayer, 2009). 
Indeed, the amount of cognitive processing of the lesson text and math-relevant visuals was 
greater in the original condition than the revised condition for students in the low prior 
knowledge group. This could be because students who saw the original visuals compensated for 
less well-designed visuals by spending more time processing the lesson information compared to 
students who saw revised visuals. For students in the middle prior knowledge group, a different 
pattern of results was noted: The amount of cognitive processing of lesson text and math-relevant 
visuals was greater in the revised condition than in the original condition. As with the cognitive 
load findings, this could be due to the revisions adding extra information. Finally, there was no 
difference in the amount of cognitive processing of the lesson text and math-relevant visuals 
between conditions for students in the high prior knowledge group (Kalyuga, 2007). It is possible 
that this group had such proficiency with the material that their overall processing was not 
affected by the revisions. Another possibility is related to the finding that, consistent with 
previous findings (Hegarty & Just, 1993; Schwonke et ah, 2009), students in the high prior 
knowledge group processed the visuals less than did students in the low and middle prior 
knowledge groups. It could be that students in the high prior knowledge group did not view the 
visuals long enough for the revisions to have an effect. 

Learning 

The effects of the revisions on learning were examined. Unlike previous work examining 
these principles in isolation with science lessons (Mayer, 2009), the revised materials did not 
promote greater learning than the original materials. Although the eye-tracking findings 
indicated an effect on cognitive load, this effect was insufficient to influence learning outcomes. 
However, information from the eye-tracking measures of amount of cognitive processing may 
explain the null effect on learning. Students in the low prior knowledge group engaged with the 
visuals and text in the original lesson more than with those in the revised lesson. It is possible 
that this additional processing compensated for the less well-designed visuals in the original 
lesson, to help students effectively learn from the original lesson. Although, according to the 
cognitive theory of multimedia learning, this additional processing would likely be extraneous 
and lead to less learning (Mayer, 2009), it appears that at least some of this additional processing 
was helpful, or at the very least not hannful, for learning. It is possible that the revisions would 
have yielded a benefit for learning for the students in the low prior knowledge group if there had 
been time-limited presentation of the materials (e.g., Amelsvoort, van der Meij, Anjewierden, & 
van der Meij, 2012). Such presentation may enhance the potential benefits of instructional design 
principles because the limited time frame may increase the need for the guidance provided by the 
labeling and color coding in the revised visuals, at least for students with low levels of prior 
knowledge. 



It is also possible that the visuals served a different role in this study than in previous 
studies. The visuals in previous studies typically represented a scientific phenomenon described 
by the text (e.g., how lightning is formed; Mayer, 2009). The nature of this study’s lesson 
content (i.e., constructing a graph) naturally prompted the integration of text and visual 
information. If the verbal representation provides instruction on how to construct the other 
representations (e.g., tables and graphs), as this lesson did, students may need less guidance 
through labels and color coding to connect the representations. 

Implications for Curriculum Development 

In this study, a “less is more” approach was embraced when revising the visuals in an 
authentic lesson (Clark & Elen, 2006). That is, three principles (signaling, contiguity, and 
coherence) were applied to reduce extraneous processing and cognitive load wherever possible 
(Mayer, 2009; Sweller et ah, 2011). However, the findings do not support the effectiveness of the 
“less is more” approach in improving student learning. One reason may be that the potential 
benefits of increasing processing in a useful way (i.e., generative processing) were not 
considered. It is possible that a “focused more is more” approach, in which students are 
prompted to engage in generative processing and materials are designed to minimize extraneous 
processing, would be more beneficial for students, (Mayer, 2014b). 

In this study, we applied a combination of three instructional design principles that shared 
common theoretical foundations in the cognitive theory of multimedia learning and cognitive 
load theory (Mayer, 2009; Sweller et ah, 2011). This is a novel approach to instructional design, 
given that these principles have generally been examined in isolation (e.g., Florax & Ploetzner, 
2010; Johnson & Mayer, 2012; Magner et ah, 2014; Mason et al., 2013; Ozcelik et ah, 2009, 
2010; Scheiter & Eitel, 2015). However, one consequence of this approach is that it is uncertain 
how the principles individually related to the findings, but it also suggests that combining 
principles may not result in additive effects. In light of our findings, it may be premature to apply 
these principles to curriculum development on a large scale. 

Theoretical Implications 

The findings from this study provide limited support for the cognitive theory of 
multimedia learning and cognitive load theory. According to the cognitive theory of multimedia 
learning, students learn more if instructional materials are designed to reduce extraneous 
processing (Mayer, 2009). According to cognitive load theory, limited working memory capacity 
is best used if cognitive load unrelated to learning (i.e., extrinsic cognitive load) is minimized 
(Sweller et al., 2011). The applications of the three instructional design principles to the revised 
visuals were all intended to reduce extraneous processing and cognitive load. The revisions 
appeared to be effective in reducing cognitive processing and cognitive load for students in the 
low prior knowledge group. However, inconsistent with these theories and previous findings 
(e.g., Florax & Ploetzner, 2010; Holsanova et al., 2009; Johnson & Mayer, 2012; Mayer et al., 
1995), the revisions did not affect student learning although they did affect cognition (similar to 
Liu et al., 2011). As previously discussed, it is possible that students in the low prior knowledge 
group engaged in compensatory cognitive processing in an effort to learn from the less well- 
designed, original visuals. 

Limitations and Future Directions 

This study applied eye-tracking methodology to understand the effects of multiple 
instructional design principles on cognitive load and the amount of cognitive processing of an 
authentic mathematics lesson. Unfortunately, because of the labor-intensive nature of eye¬ 
tracking data collection and the curvilinear patterns with prior knowledge, there was insufficient 



power to directly examine relationships between the eye-tracking measures reported and 
learning. Future work could better elucidate our understanding of how different representations 
are processed in relation to learning. Such work could detect relationships by targeting a specific 
prior knowledge group—individuals with low prior knowledge may be of particular importance 
given that instructional design principles appear to have the most influence for such individuals 
(Mayer, 2001). Such work might also consider a single instructional principle at a time. In 
addition, in future studies, less authentic, but more controlled placement of visuals may allow for 
a more fine-grained understanding of relations between eye-tracking measures and learning. The 
use of controlled placement of visuals would also be conducive to eye-tracking measures not 
reported in this chapter, such as transitions between visuals and text. 

CONCLUSION 

In this study, visuals were revised in a mathematics lesson based on instructional design 
principles to improve the integration of text and visual information, and the effects on cognitive 
load, cognitive processing, and learning were examined. Eye-tracking methodology was used to 
examine the effects of the revisions on viewing the text and the math-relevant visuals. Consistent 
with the cognitive theory of multimedia learning (Mayer, 2014a) and cognitive load theory 
(Sweller et al., 2011), the revisions appeared to reduce cognitive processing and cognitive load 
for students with low levels of prior knowledge, though they did not affect learning. . 

Taken together, the findings emphasize the complexity of applying instructional design 
principles to authentic lessons. Although many instructional design principles are framed as “one 
size fits all,” the findings indicate that their effects on student cognition may depend on students’ 
level of prior knowledge. This study illuminates the usefulness of eye-tracking data in 
understanding cognitive load and cognitive processing among different groups of students 
learning from multiple representations. 
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KEY TERMS AND DEFINITIONS 

Cognitive load: The amount of effort one exerts while processing infonnation in working 
memory (Sweller et ah, 2011). 


Cognitive processing: The amount of thinking in which one engages for a particular task or 
topic. 



Coherence principle: Learning is facilitated when interesting, but irrelevant infonnation is 
removed (Mayer, 2009). 

Contiguity principle: Learning is facilitated when materials have relevant information in 
different representations in close physical proximity (Mayer, 2009). 

Eye-mind assumption: The concept that the eye fixates on what the mind is processing (Just & 
Carpenter, 1980). 

Instructional design principles: Approaches for the arrangement of learning materials based on 
theoretical understanding of human cognition. Also called cognitive principles or evidence- 
based principles. 

Signaling principle: Learning is facilitated when materials have cues to important infonnation 
(Mayer, 2009). 



